Strategies for fast convergence in semiotic dynamics 
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Abstract 

Semiotic dynamics is a novel field that studies how semiotic 
conventions spread and stabilize in a population of agents. 
This is a central issue both for theoretical and technological 
reasons since large system made up of communicating agents, 
like web communities or artificial embodied agents teams, are 
getting widespread. In this paper we discuss a recently in- 
troduced simple multi-agent model which is able to account 
for the emergence of a shared vocabulary in a population of 
agents. In particular we introduce a new deterministic agents' 
playing strategy that strongly improves the performance of 
the game in terms of faster convergence and reduced cogni- 
tive effort for the agents. 

Introduction 

Imagine a population of artificial embodied agents explor- 
ing an unknown environment. One of the first tasks they 
should face is exchanging informations about their discov- 
eries. In particular, when a new 'thing' is met, they should 
be able to agree on its identification. If the agents were en- 
dowed with short range communication systems only, the 
agreement would take place locally, and, in a second time, 
should hopefully become global. But how could this hap- 
pen? A first hypothesis is that they could agree on the ge- 
ographical location of the object and everybody could go 
and see it. This is of course very costly, and definitely not 
efficient since each new finding of the same object would 
require the same spending procedure. Moreover it would 
require a mechanism of global coordination, which is not al- 
ways available. Instead, it would be desirable that each agent 
could recognize the object the first time it sees it and then as- 
sign a true 'name' to it. The global agreement on the name 
would then allow for a great saving of time and would also 
be crucial for the birth of a communication system among 
agents. This kind of problems inspired the well known 
"Talking Heads Experiment" in which embodied software 
agents were shown to be able of bootstrapping a shared set 
of semiotic conventions (Steels, 1999 1. 

In the following, we shall assume that our agents are en- 
dowed with the necessary tools needed to recognize and 
physically deal with an object and we shall concentrate on 



the dynamics that leads to the obtention of a shared set 
of conventions in a population. This is the general prob- 
lem investigated by the field of Semiotic Dynamics, ac- 
cording to which language is an evolving and self orga- 
nized system (Steels, 2000). The term evolving must not 
be misleading. The evolution of language across genera- 
tions is a well investigated aspect, and multi-agent modeling 
has already proved to be a powerful tool for its investiga- 
tion ( |Hurford, 1989||Nowak et al., 1999| l. In our context, on 
the other hand, the focus is on much shorter timescales, so 
that we are not dealing with transmission across generations 
nor, more in general, Darwinian concepts. The issue of the 
self-organization of language on fast temporal scales is, of 
course, of the outmost importance and generality. Besides 
artificial systems, where it is obvious that the agreement has 
to take place rapidly, it concerns human dynamics, too. In 
particular the user based tagging systems presently spread- 
ing on the web (such as del.icio.us or flickr.com), where 
users manage tags to share and categorize informations, of- 
fer wonderful examples of self-organized communication 
systems. Gaining hints into population-scale semiotic dy- 
namics is then important for a twofold reason. On the one 
hand it is necessary to interpret and understand presently oc- 
curring phenomena, and on the other hand it can be very im- 
portant to provide indications for the design of large scale 
technological systems. 



In this paper we focus on a recently introduced multi- 
agent model ( jBaronchelh et al.,"^0 05 ), inspired by the so- 
called Naming Game ( Steels ~995| l, which, though being 
very simple in its definition, is able to account for the birth of 
a shared set of conventions in a population. We investigate 
also how the properties of the system change with the popu- 
lation size. We then study how a modification of the rules of 
the original model, from random to deterministic strategies, 
allows to improve the performance of the population: both 
the time required for the consensus to emerge and the agents 
memory requirements are indeed substantially lowered. 



A simple model of semiotic dynamics 

Let us consider a population of N agents which perform pair- 
wise games in order to agree on the name to assign to a sin- 
gle object. Each agent is characterized by his inventory or 
memory, i.e. a list of name-object associations that is empty 
at the beginning of the process and evolves dynamically dur- 
ing time. At each time step, two agents are randomly se- 
lected, one to play as speaker, the other one as hearer, and 
interact according to the following rules: 

• The speaker has to transmit a name to the hearer. If his 
inventory is empty, he invents a new word, otherwise he 
selects randomly one of the names he knows; 

• If the hearer has the uttered name in his inventory, the 
game is a success , and both agents delete all their words 
but the winning one; 

• If the hearer does not know the uttered word, the game is 
a failure , and the hearer inserts the word in its inventory, 
i.e. he learns it. 

A remark concerning the presence of a single object is 
in order. From a linguistic point of view it is equivalent to 
the rather strong assumption of preventing homonymy, thus 
making different objects independent. This simplification 
allows for a strong reduction in the complexity of the model 
and, moreover, does not seem so drastic when thinking of 
artificial agents that assign randomly extracted real numbers 
to new objects. 

Another point concerns the difference with some other 
well-known models of opinion and consensus formations. 
In Axelrod's model ( Ax elrod, 1997) , each agent is endowed 
with a vector of opinions, and can interact with other agents 
only if their opinions are already close enough; in Sznajd's 
model (Sznajd-Weron and Sznajd, 2000 1 and in the Voter 
model (Krapivsky, 1992 1, the opinion can take only two dis- 
crete values, and an agent takes deterministic ally the opin- 
ion of one of his neig hbors. Also in (Deffuant et al., 2000 1, 
the opinion is modeled as a unique variable and the evolu- 
tion of two interacting agents is deterministic. In the Nam- 
ing Game model on the opposite, each agent can potentially 
have an unlimited number of possible discrete states (or 
words, names) at the same time, accumulating in his mem- 
ory different possible names for the object: the agents are 
able to "wait" before reaching a decision. Moreover, each 
dynamical step can be seen as a negotiation between speaker 
and hearer, with a certain degree of stochasticity. 

To understand the behavior of the system (see 
also jBaronchelh et al., 2005| l), we report in Figure ^ 
three curves obtained averaging several runs of the process 
in a population of Af = 2 x 10 3 agents. They represent 
the evolution in time of the total number of words present 
in the system N w (t), which quantifies the total amount of 
memory used by the process, of the number of different 
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Figure 1 : Process evolution - The total number of words in 
the system, N w (f ) (or total memory used), the number of dif- 
ferent words, Nd(t), and the success rate, 5(f), are plotted as 
a function of time. The final convergence state is character- 
ized by the presence of the same unique word in the inven- 
tories of agents. Thus, at the end of the process, we have 
N w (t) = N and N c i{t) = 1, while the probability of a success 
is equal to 1 (S(t) = 1). The curves have been obtained av- 
eraging over 300 simulation runs for a population of 2 x 10 3 
agents. 



words, Nd(t), and of the success rate, 5(f), defined as the 
probability of a successful interaction between two agents at 
time f . The first thing to be noted is that the system reaches 
a final convergence state in which all agents have the same 
unique word, i.e. a final proto-communication system has 
been established. It is thus interesting to proceed with 
a more detailed analysis of how this final state of global 
communication emerges from purely binary interactions. 

The process starts with a trivial phase in which the inven- 
tories are empty, so that the invention process is dominat- 
ing and N/2 different words are created on average. This 
rapid transient is followed by a longer period of time in 
which most interactions are unsuccessful (5(f) ~ 0), and the 
sizes of inventories keep growing. However the amount of 
memory used does not increase indefinitely, since correla- 
tions are progressively built up among inventories and in- 
crease the probability of successful interactions. In partic- 
ular, the N w (t) curve exhibits a well identified peak, whose 
height and occurrence time are important parameters to de- 
scribe the process. Slightly after the peak, there is a quite 
abrupt transition from a disordered state in which communi- 
cation among agents is difficult to a nearly optimal situation, 
which is captured by a jump of the success rate curve. The 
process then ends when the convergence state {N c i(t) = 1, 
N w (t) — N) is reached. Finally, it is worth noting that the de- 
veloped proto-communication system is not only effective 
(each agent understands all the others), but also efficient (no 
memory is wasted in the final state). 



I 

- . t 

max 

I . t 

j conv 

* cliff 

1 — to* N , a= 1.5 


1 ^1 









Figure 2: Scaling with the population size N. In the up- 
per graph the scaling of the peak and convergence time, tma X 
and t conv , is reported, along with their difference, tdiff. All 
curves scale with the power law N 1 ' 5 . Note that t CO m> and 
tdiff scaling curves present characteristic log-periodic oscil- 
lations. The lower curve shows that the maximum number of 
words (peak height, N"' ax = N w (t max )) obeys the same power 
law scaling. 



To gain a deeper comprehension of the process it is im- 
portant to investigate how the main features scale with the 
system size N. In particular, it is relevant to know how the 
agents' cognitive effort in terms of memory and the time re- 
quired to reach the final state depend on the population size. 
The global memory used is maximum when the number of 
words is the highest, i.e. at the peak of the N w (t) curve. The 
scaling of the peak time t max and height N" MX = N w (t max ) are 
therefore studied, together with the convergence time t com , 
in Figure[2] It turns out that all these quantities follow power 
laws: tmax ~ N a , t conv ~ N$, t diff = (t conv - t max ) ~ N 5 , and 
N™ x ~ with exponents a«P«y«S«1.5. More pre- 
cisely, each agent accumulates at the peak a number of or- 
der N°' 5 different words, which means that the necessary 
memory per agent grows notably when the system size is 
increased. 

Let us mention that the values for a and y can in fact 
be understood through simple analytical arguments. As- 
sume indeed that, at the maximum, the average number of 
words per agent scales as cN a . The probability for an agent 
chosen as speaker to utter a specific word is l/(cN"), and 
the probability that the hearer already possesses this word 
is cN a I (N /2). The balance of unsuccessful interactions 
(which increase N w by one unit) and successful ones (which 
decrease N w by 2cN") can then be written as: 
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At the maximum, dN ^ = 0, so that the only possibility is 
a = 1/2. Similar arguments can be applied to the derivation 



of the exponent for the time of the peak. It is important to 
stress that these analytical results can be obtained thanks to 
the simplicity of the microscopic interaction rules. 

In summary, the time to convergence grows quite fast as a 
function of the system size, and the necessary memory used 
by each agent also diverges when N grows. A natural and 
important question is therefore whether it is possible to im- 
prove the performance of the system. More precisely, a ma- 
jor challenge would be to improve the population-scale per- 
formances of the process without loosing the simplicity of 
the microscopic rules, which is the precious ingredient that 
allows for in-depth investigations of global-scale dynamics. 
We will address this problem in the next section. 

Smart Strategy 

In the model described in the previous section, agents, when 
playing as speakers, extract randomly a word in their in- 
ventories. This feature, along with the drastic deletion 
rule that follows a successful game, is the distinctive trait 
of the model. Indeed, most of the previously proposed 
models of semiotic dynamics prescribe that a weight is 
associated to each word in each inventory: this weight 
determines its probability of being chosen (see, for in- 
stance (Lenaer ts et al., 20 05 1, and references therein). As 
a natural consequence the effect of a successful game con- 
sists in updating the weights, rewarding the weight associ- 
ated with the winning word and possibly reducing the others. 
Such sophisticated structures can in principle lead to faster 
convergence, but make the models more complicated, com- 
promising the possibility of a clear global scale picture of 
the convergence process. 

In order to maintain the simplicity of the dynamical rules, 
it seems natural to alter the purely stochastic selection rule 
of the word chosen by the speaker. In the model previously 
described, all the words of a given agent's inventory share a 
priori the same status. However, a simple parameter to dis- 
tinguish between them is their "arrival time", i.e. the time 
at which they enter in his inventory. In particular two words 
are easily distinguished from the others: the last recorded 
one and the last one that gave rise to a successful game, 
i.e. the first that was recorded in the new inventory gen- 
erated after the successful interaction. Natural strategies to 
investigate consist therefore in choosing systematically one 
of these particular words. We shall refer to these strategies 
as "play-last" and "play-first" respectively. Other selection 
rules are of course possible but would be either more com- 
plicated or more artificial. 

The scaling behavior of the model when the "play-last" 
strategy is adopted is very interesting (data not shown). The 
peak time and height scale respectively as t max ~ N a with 
a « 1.3 and N" mx ~ A^ with y w 1.3, i.e. the used memory 
is reduced, while the convergence time scales as t conv ~ N$ 
with p w 2.0. At the beginning of the process, playing 
the last registered word creates a positive feedback that en- 




Figure 3: Play-smart strategy - scaling with the population 



size N. Top - For the time of the peak t max ~ N a , a ~ 1.3, 
while for the convergence time we have t com ~ aN a + bN 5 
with 8 ~ 1.3, 8 » 1.0. Bottom - the maximum number of 
words scales as N" mx ~ with yw 1.3. The "play-smart" 
rule gives rise to a more performing process, from the point 
of view of both convergence time and memory needed. 



hances the probability of a success. In particular a circulat- 
ing word has more probabilities of being played than with 
the usual stochastic rule, thus creating a scenario in which 
less circulating words are known by more agents. On the 
other hand the "last in first out" approach is highly inef- 
fective when agents start to win, i.e. after the peak. In 
fact, the scaling t com ~ can be explained through sim- 
ple analytical arguments. Let us denote by N a the number 
of agents having the word "a" as last recorded one. This 
number can increase by one unit if one of these agents is 
chosen as speaker, and one of the other agents is chosen as 
hearer, i.e. with probability N a /N X (1 —N a /N); the prob- 
ability to decrease N a of one unit is equal to the probability 
that one of these agents is a hearer and one of the others is 
a speaker, i.e. (1 —N a /N)N a /N. These two probabilities are 
perfectly balanced so that the resulting process for the den- 
sity p a = N a /N can be written as an unbiased random walk 
(with actually a diffusion coefficient p„(l — p a )/N 2 ); it is 
then possible to show that the time necessary for one of the 
p a to reach 1 is of order A^ 2 . In summary, in this framework it 
is much more difficult to bring to convergence all the agents, 
since each residual competing word has a good probability 
of propagating to other individuals. 

The "play-first" strategy, on the other hand, leads to a 
faster convergence. Due to a sort of arbitrariness in the 
strategy before the first success of the speaker, the peak re- 
lated quantities keep scaling as in the usual model, so that 
tmax ~ N a and N'™ x ~ N y with a « y ks 1 .5 (data not shown). 
This seems natural, since playing the first recorded word is 
essentially the same as extracting it randomly when most 
agents have only few words. In fact, in both cases no vir- 
tuous correlations or feedbacks are introduced between cir- 
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Figure 4: "Play-smart" strategy - Fraction V(t) of agents 
who have played at least one successful game at time t. The 
transition between the initial condition, in which all agents 
play the last heard word, to the final one, in which agents 
play the word which took part in their last successful inter- 
action, is continuous. The growth gets faster after the peak 
of N w . 



culating and played words. However, playing the last word 
which gave rise to a successful interaction strongly improves 
the system-scale performances once the agents start to win. 
In particular it turns out that for the difference between the 
peak and convergence time we obtain (t com , — t max ) ~ N 5 
with 8 « 1.15 (data not shown), so that the behavior of the 
convergence time is the result of the combination of two dif- 
ferent power law regimes, i.e. t conv ~ aN a + bN s . On the 
other hand, the stochastic rule leads to (t com , — t max ) ~ A^ 1,5 
as shown in Figure This means that the "play-first" strat- 
egy is able to reduce the time that the system has to wait be- 
fore reaching the convergence, after the peak region. This 
seems the natural consequence of the fact that successful 
words increase their chances to be played while suppress- 
ing the spreading of other competitors. 

In summary, we have seen that, compared to the usual ran- 
dom extraction of the played word, the "play-last" strategy 
is more performing at the beginning of the process, while 
the "play-first" one allows to fasten the convergence of the 
process, even if it is effective only after the peak of the total 
number of words. It seems profitable, then, to define a third 
alternative strategy which results from the combination of 
the two we have just described. The new prescription, which 
we shall call "play-smart", is the following: 



— ► If the speaker has never took part in a successful game, he 
plays the last word recorded; 

-> Else, if the speaker has won at least once, he plays the last 
word he had a communicative success with. 

The first rule will thus be applied mostly at the beginning, 
and as the system evolves, the second rule will be progres- 
sively adopted by more and more agents. Since the change 
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Figure 5: Success rate curves S(t) for the various strate- 
gies: stochastic, "play-last", "play-first" and "play-smart". 
At the beginning of the process the stochastic and "play- 
first" strategies yield similar success rates, but then the deter- 
ministic rule speeds up convergence. On the other hand also 
the "play-smart" and the "play last" evolve similarly at the 
beginning, but the latter reaches the final state much earlier 
through a steep jump. It is worth noting that for three strate- 
gies the S(t) curves present a characteristic S — shaped be- 
havior, while in the "play-last" one the disorder-order tran- 
sition is more continuous (see inset in the top figure). All 
curves, both for Af = 10 3 and Af = 10 4 , have been generated 
averaging over 3 x 10 3 simulation runs. 



in strategy is not imposed at a given time, but takes place 
gradually, in a way depending of the evolution of the sys- 
tem, such a strategy has also the interest of being in some 
sense self-adapting to the system's actual state. In Figure[3] 
the scaling behaviors relative to the "play-smart" strategy 
are reported. Both the height and time of the maximum fol- 
low the scaling of the "play-last" strategy: t max ~ N a and 
N™ ax ~ with a « y ~ 1.3. The convergence time, on 
the other hand, scales as a superposition of two power laws: 
t conv ~ aN a + bN 5 with a w 1.3,8 « 1.0. Thus, the global 
behavior determined by the "play-smart" modification is in- 
deed less demanding in terms of both memory and time. In 
particular, while the lowering of the peak height yields in 
fact a slower convergence for the "play-last" strategy, the 
progressive self-driven change in strategy allows to fasten 
the convergence further than for the "play-first" strategy. 

It is also worth studying how the transition between the 
initial situation in which most agents play the last recorded 
word to that in which they play the last successful word 
takes place in the "play-smart" strategy. In other words we 
want to study the probability V(t) of finding an agent that 
has already been successful in at least one interaction at a 
given time. Results relative to a population of 10 4 agents 
are shown in Figure^ Interestingly, the transition from the 
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Figure 6: Total number of words N w (t ) for the various strate- 
gies: stochastic, "play-last", "play-first" and "play-smart". 
Due to different scaling behaviors of the process, differences 
become more and more relevant for larger N (top figure: 
N = 10 3 ; bottom figure: N = 10 4 ). The "play-smart" ap- 
proach combines the advantages of "play-last" and "play- 
first" strategies. 



initial situation to the final one is continuous, and there is a 
sudden speeding up after the peak. 

Finally, in order to have an immediate feeling of what dif- 
ferent playing word selection strategies imply, we report in 
Figures [5] and [6] the success rate S(t) and the total number 
of words, N w (t) relative to the four strategies described pre- 
viously, for two different sizes. The "play-first" and "play- 
smart" curves exhibit the same "S-shaped" behavior for S(t) 
as in the case of the stochastic model, while the "play-last" 
rule affects qualitatively the way in which the final state is 
reached. Indeed, in this case the transition between the ini- 
tial disordered state and the final ordered one is more contin- 
uous (see the inset in the top figure). Moreover, Figure[6]il- 
lustrates that the choice of the strategy has substantial quan- 
titative consequences for both necessary memory and time 
needed to reach convergence, even if the changes in scaling 
behavior could at first appear rather limited (from A^ 1 5 to 
N 13 ). In particular, the "play-smart" strategy, which adapts 
itself to the state of the system, leads to a drastic reduction of 
the memory and time costs and thus to a dramatic increase 
in efficiency. 

Conclusion 

In conclusion, we have discussed a multi-agent model of 
Semiotic Dynamics which is able to describe the conver- 
gence of a population of agents on the use of a particular 
semiotic convention (a name to assign to an object, in our 
case). The model relies on very simple microscopic interac- 
tion rules, thus being appropriate for accurate global scale 
investigations. We have then shown that the modification of 
the rule followed by agents to select the word to be transmit- 



ted gives rise to a process which is less demanding in terms 
of agents memory usage and leads to a faster convergence, 
too. Due to the possible utility of Semiotic Dynamics mod- 
els for the design of technological systems, we believe that 
the findings presented here are not only intrinsically interest- 
ing from a theoretical point of view, but can also be relevant 
for applications. 
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